🕷️️ Job Radar • SCRAPING

freelancer.com 2026-04-14 🟡

🔹 Consolidate Multi-Source Data into Uniform Format
👤 Client: 🇸🇦 Riyadh, Saudi Arabia Member since 2026-04-11
💰 Price: $20 / hr Average bid
🚩 Problem: Convert diverse datasets from websites, databases, and social media into a single, uniform format for internal reporting.
📦 Existing: Raw exports/downloads from various sources, field-mapping notes.

Specifications:

[Target] Consolidated CSV file with consistent structure and encoding.
[Method] Data extraction, transformation, and loading (ETL) process.
[UI/UX] Not applicable; command-line or API interface preferred for automation.
[Stack] Python with libraries like Pandas, SQLAlchemy, and BeautifulSoup. SQL for database interactions.
[Security] Ensure data privacy during transfer and processing; use secure connections where necessary.
[Format] CSV file with headers matching the provided field map.

Workflow:

1. Extract data from websites using web scraping techniques (BeautifulSoup or Selenium).
2. Load database dumps into a local database for easy querying (e.g., PostgreSQL, MySQL).
3. Retrieve social-media insights via API calls and store in the same database.
4. Transform all datasets to a common schema using Pandas DataFrame operations.
5. Validate data integrity and consistency across sources.
6. Export transformed data into a single CSV file with headers as per field map.